Features generally relate to the field of data storage, and, more particularly, to atomicity using write-back caching in a distributed storage system.
The creation and storage of data has proliferated in recent years. Accordingly, there are a number of techniques and mechanisms that facilitate efficient and cost effective storage of large amounts of data. For example, a cluster network environment of nodes may be implemented as a data storage system to facilitate the creation, storage, retrieval, and/or processing of data. Such a data storage system may be implemented using a variety of storage environments, such as a network-attached storage (NAS) environment, a storage area network (SAN), a direct-attached storage environment, and combinations thereof. The foregoing data storage systems may comprise one or more data storage devices configured to store data within data volumes or containers.
Each data container can be serviced by a particular computer or node. Each data container can support a defined number of inodes, wherein each inode can represent one file or directory. These separate containers have limits regarding their size (derived from how many disks the computer can support), how much traffic can be served (derived from the processor, memory and network bandwidth available on the computer), how many discrete files can be stored (often limited by choice of storage format, such as using 32-bit quantities for inode numbers), etc. These container-derived limits are often very hard to raise. Raising these container limits can require significant investment in new hardware, for example, or an extensive software redesign that would require substantial investment.